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Abstract 

This  paper  describes  a  defensible  process  for  selecting  the  combinations  of 
feasible  and  viable  technologies  that  have  the  highest  potential  to  mitigate  the 
current  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  capability  gaps 
for  helicopters  operating  in  a  Degraded  Visual  Environment  (DVE).  The  process 

for  selecting  these  combinations  is  an  adaptation  of  traditional  Systems 

This  paper  has  not  been  presented,  published,  nor  submitted  for  publication  prior  to  this 
submission. 
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Engineering  techniques,  including  Morphological  Analysis,  Quality  Function 
Deployment,  and  Utility  Theory.  After  they  are  identified,  the  Army  intends  to 
examine  the  inclusion  of  the  top  combinations  (or  a  subset  thereof)  into  ground, 
simulation,  and  flight  test  activities  in  order  to  better  understand  their  ability  to 
mitigate  the  gaps.  Additionally,  if  technologies  present  a  unique  capability  of 
interest,  the  specific  technology  may  be  examined  further. 


AAD 

AATD 

AMRDEC 

ADS 

AFDD 

AHP 

BAR 

CERDEC 

DSE 

DVE 

EO/IR 

FNA 

GTRI 

LADAR 

LIDAR 


Nomenclature 

Aviation  Development  Directorate 
Aviation  Applied  Technology  Directorate 

Aviation  &  Missile  Research  Development  &  Engineering  Center 

Aeronautical  Design  Standard 

Aerofiightdynamics  Directorate 

Analytic  Hierarchy  Process 

British  Aerospace  Engineering 

U.S.  Army  Communications-Electronics  Research,  Development  and 

Engineering  Center 

Decision  Support  Environment 

Degraded  Visual  Environment 

Electro-Optic/Infra-Red 

Functional  Needs  Analysis 

Georgia  Tech  Research  Institute 

Laser  Radar  (analogous  to  LIDAR) 

Light  Detection  and  Ranging  (analogous  to  LADAR) 
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MADM 

Multi-Attribute  Decision  Making 

MMWR 

Millimeter-Wave  Radar 

MTE 

Mission-Task  Element 

NOE 

Nap  of  the  Earth 

NVESD 

Night  Vision  &  Electronics  Sensors  Directorate 

OPSEC 

Operational  Security 

PEO 

Program  Executive  Office 

QFD 

Quality  Function  Deployment  (a.k.a.  House  of  Quality) 

SME 

Subject  Matter  Expert 

SNC 

Sierra  Nevada  Corporation 

SSDD 

System  Simulation  &  Development  Directorate 

TRADOC 

U.S.  Army  Training  and  Doctrine  Command 

UCE 

Usable  Cue  Environment 

UH 

Utility  Helicopter 

Introduction 

In  July  2012,  the  U.S.  Army  began  the  AMRDEC  DVE  Mitigation  Program  in  order 
to  execute  a  synchronized,  collaborative  effort  across  the  AMRDEC  Science  & 
Technology  community  to  parametrically  define  flight  control  system,  cueing,  and  sensor 
combinations  which  enable  the  “closing”  of  the  current  TRADOC  capability  gaps  that 
pertain  to  rotorcraft  operations  in  degraded  visual  environments.  Additionally,  through 
analysis,  simulation,  and  ground  and  flight  test,  the  stakeholders  on  the  team  are 
exploring  the  tradespace  to  assist  PEO  Aviation  in  making  informed  decisions  on  future 
materiel  upgrades  and  potential  programs  of  record.  This  paper  focuses  on  the  initial 
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analysis  efforts  to  investigate  the  contributions  of  the  various  technologies  to  closing  the 
TRADOC  DVE  capability  gaps.  The  capstone  event  of  the  AMRDEC  DVE  Mitigation 
Program  will  be  demonstration  flights  at  Yuma  Proving  Ground  in  FY16  involving 
modernized  flight  control  laws,  multi-resolution  sensor  fusion,  and  varied  cueing 
combinations. 

Over  the  past  10  years,  U.S.  Army  rotorcraft  accidents  due  to  DVE  have  resulted  in 
87  aircraft  accidents  (Class  A  and  B)  resulting  in  108  fatalities  and  over  $880M  in 
material  losses.  Many  of  these  losses  were  due  to  operations  in  “Brownout”  (see  Fig.  1). 


Fig.  1.  Rotorcraft  operations  in  brownout. 

Although  it  may  be  accentuated  due  to  the  current  theater  of  combat  operations,  DVE 
is  more  than  just  “Brownout”  (helicopter  induced  dust  cloud  due  to  the  downwash  of  the 
rotor  system).  Categories  of  DVE  have  been  identified  (Fig.  2)  and  accompanying  data 
for  potential  future  specification  verification  (e.g.,  “levels”  of  the  different  categories)  has 
been  identified  (Fig.  3).  Data  exists  for  the  other  DVE  categories,  but  only  the  brownout 
environment  is  illustrated  and  the  accompanying  analysis  is  primarily  focused  on 
brownout.  However,  a  sensitivity  analysis  was  also  rapidly  conducted  to  present  a 
snapshot  of  results  when  the  other  environments  in  Fig.  2  are  considered.  Pursuit  of 
materiel  solutions  will  allow  for  not  only  the  safe  and  efficient  operation  of  rotorcraft  but 
also  expand  the  ability  of  commanders  to  deploy  their  rotorcraft  aviation  assets  when  the 
weather  is  well  below  minimum  visual  meteorological  conditions.  This  objective  is  the 
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basis  for  the  mantra  of  the  AMRDEC  DVE  Mitigation  Program  team:  “Own  the 
Weather”! 


AJftmi  induced  DVE  Aircraft  tndependentDegnided  Visual  Environments 


Fig.  2.  Categories  of  DVE. 
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Fig.  3.  Characterization  of  the  brownout  environment  utilized  for  this  study. 

Working  with  TRADOC,  the  definition  of  DVE  was  formalized  for  use  by  the  Army 
rotorcraft  community  in  2012  as  the  following: 

Reduced  visibility  of  potentially  varying  degree,  wherein  situational  awareness 
and  aircraft  control  cannot  be  maintained  as  comprehensively  as  they  are  in 
normal  visual  meteorological  conditions  and  can  potentially  be  lost. 
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Three  Legs  of  the  DVE  Stool 

Army  officials  view  potential  DVE  system  solutions  as  being  comprised  of  three  legs. 
In  order  to  expand  U.S.  warfighting  capability  and  conduct  deliberate  combat  operations 
with  Level  1  Handling  Qualities  (i.e.,  satisfactory,  no  improvement  required)  in  DVE,  all 
legs  are  necessary  as  no  combination  missing  one  or  two  leg(s)  is  sufficient.  The  three 
legs  are: 

1.  Improved  Flight  Controls:  Involves  enhancing  the  existing  flight  controls  systems 
and/or  laws  and  handling  characteristics  to  assist  the  pilot  in  managing  workload 
when  vision  or  situational  awareness  is  challenged  or  obscured. 

2.  Sensors'.  Includes  technologies  that  allow  “see-through”  capability  when  the 
environments  in  Fig.  2  are  encountered. 

3.  Cueing;.  Comprises  symbology,  aural,  or  tactile  indicators  providing  information  to 
the  pilot  to  reference  aircraft  state  and  potentially  provide  guidance  for  executing  a 
mission  task  element  (MTE)  such  as  landing,  take-off,  or  hover. 

Stakeholders  involved  in  exploring  these  three  legs  of  the  DVE  stool  include  the 
AMRDEC  Aviation  Development  Directorate  [Aviation  Applied  Technology  Directorate 
(AATD)  and  the  Aeroflightdynamics  Directorate  (AFDD)],  CERDEC  Night  Vision  & 
Electronics  Sensors  Directorate  (NVESD),  AMRDEC  System  Simulation  & 
Development  Directorate  (SSDD)  and  the  Georgia  Tech  Research  Institute  (GTRI). 

AMRDEC  DVE  Mitigation  Program  Objectives 


The  AMRDEC  DVE  Mitigation  Program  has  four  main  objectives: 
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1 .  Develop  an  experimental  plan  using  Design  of  Experiment  techniques  focused  on 
examining  contributions  of  different  materiel  technologies  with  regards  to  the  eight 
TRADOC  DVE  capability  gaps. 

2.  Provide  detailed  updates  to  pertinent  specifications  (e.g.,  ADS-33  (Ref.  1),  MIL- 
STD-1787)  with  quantitative  pilot-vehicle  DVE  operational  performance  data. 

3.  Baseline  control  system,  sensor,  and  cueing  technologies  for  possible  off-ramp 
infusions  into  PEO  Aviation  programs  of  record. 

4.  Conduct  and/or  participate  in  flight  demonstrations  with  numerous  flight  control, 
sensor(s)  and  cueing  configurations. 

Flight  Controls 

Overall,  the  Army  considers  the  DVE  challenge  to  rotorcraft  to  be  a  handling 
qualities  problem  and  considers  flight  control  augmentation  to  be  a  necessary  part  of  the 
solution.  The  AMRDEC  DVE  Mitigation  Program  will  examine  control  law  upgrades, 
commonly  referred  to  as  MCLAWS,  or  Modernized  Control  Laws,  that  are  exercised  in 
the  flight  control  computer  (see  Fig.  4).  These  upgrades  may  also  include  physical 
hardware  (e.g.  collective  trim  servo)  and  will  result  in  improved  handling  qualities  in 
accordance  with  the  Army’s  handling  qualities  specification:  Aeronautical  Design 
Standard  33,  or  ADS-33  (Ref.  1).  The  addition  of  advanced  inner  and  outer  loop  “modes” 
in  the  flight  control  system  (e.g.,  attitude  command/attitude  hold,  airspeed  hold,  height 
hold,  heading  hold,  and  position  hold)  will  ultimately  reduce  the  pilot’s  workload  when 
operating  in  conditions  of  reduced  visibility.  With  a  more  stable  platform,  the  crew  will 
then  be  better  able  to  assess  other  concerns,  e.g.,  approaching  terrain  and  obstacles, 
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potential  landing  area  suitability  and  maintain  improved  situational  awareness  of  overall 


aircraft  state. 


Fig.  4.  Flight  control  system  example. 


Sensors 

In  the  past,  numerous  sensor  technologies  have  been  studied  and  demonstrated,  e.g., 
LADAR  (Fig.  5),  Millimeter  Wave  Radar  (Fig.  6)  and  Infrared  (Fig.  7).  Overall,  the 
“DVE  community”  has  a  good  understanding  of  the  enhancing  capabilities  as  well  as  the 
shortcomings  of  each  type  of  sensor,  primarily  in  terms  of  penetration/range  vs. 
resolution  in  different  degraded  visual  environments.  In  order  to  achieve  “see-through” 
capability  in  as  many  of  the  categories  of  DVE  (Fig.  2)  as  possible,  the  AMRDEC  DVE 
Mitigation  Program  is  focusing  on  sensor  fusion.  A  major  part  of  the  effort  is  better 
understanding  the  difficulty  in  integrating  multi-resolution  data  sources  (as  well  as  a 
priori  data)  and  balancing  the  trade-offs  in  imagery  resolution  that  are  required  in 
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presenting  a  cohesive  “scene”  during  different  flight  modes  (e.g.  low  level,  contour,  or 
nap  of  the  earth  (NOE)  flight). 


Fig.  5.  Bums  Engineering  Corporation  LADAR  system  as  installed  on  ADD-AFDD  EH- 
60L  executing  brownout  landings  at  Yuma  Proving  Ground  in  June  2013. 


Fig.  6.  Sierra  Nevada  Corporation  94  GHz  millimeter  wave  radar  system  as  installed  on 
ADD-AATD  EH-60L  executing  flight  test  at  Fort  Eustis  in  May  2012. 


Fig.  7.  Raytheon  Infrared  Distributed  Aperture  System  (DAS)  as  installed  on  CERDEC- 
NVESD  UH-60A  executing  flight  test  at  Fort  Belvoir  in  March  2012. 
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Cueing 


The  AMRDEC  team  intends  to  execute  two  crew  station  working  groups  over  the 
course  of  the  program  to  explore  the  use  of  3D  (Fig.  8)  and  2D  (Fig.  9)  symbology  cueing 
overlaid  upon  imagery  on  both  a  panel  mounted  as  well  as  a  helmet  mounted  pilot 
medium.  Individual  sensor  inputs  will  be  emulated  in  the  AMRDEC  SSDD  UH-60L 
simulator  at  Redstone  Arsenal,  Alabama.  Additional  cueing  elements  such  as  tactile  and 
aural  (Fig.  10)  will  be  investigated  to  understand  their  potential  value  as  a  part  of  an 
integrated  DVE  system  solution. 


Fig.  9.  2D  symbology  cueing  example  (overlaid  on  94  GHz  millimeter  wave  radar 
imagery  with  coincident  day  TV  camera  scene  shown  on  right). 
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Fig.  10.  Tactile  and  aural  technology  systems  views. 

Overall,  the  AMRDEC  DVE  Mitigation  Program  is  oriented  toward  examining  the 
combinations  of  required  technologies  that  will  give  Army  rotorcraft  pilots  the  advantage 
on  the  battlefield.  In  total,  this  integrated  three  pronged  approach  to  a  DVE  system 
solution  is  aimed  at  increasing  aircrew  safety  and  survivability  while  also  helping  to 
provide  them  every  conceivable  tactical  and  operational  advantage.  Through  the  use  of  a 
defensible  decision  support  environment  (DSE),  the  Army  intends  to  take  the  first  steps 
in  executing  a  coherent  test  program  to  better  understand  the  trade-offs  between  the 
available  technologies  that  populate  the  three  legs  of  the  DVE  stool. 


Methodology 


This  methodology  intends  to  produce  a  defensible  DSE  for  selecting  the  most  suitable 
set  of  technologies  to  mitigate  the  capability  gaps  for  safe  operations  of  rotorcraft  in 
DVE.  A  decision  is  considered  to  be  defensible  if  it  can  be  justified  with  a  rational  and 
relevant  argument.  Rationality  requires  the  process  to  be  transparent,  traceable,  and 
logical,  whereas  relevancy  requires  the  data  supporting  the  process  to  be  accurate, 
current,  and  derived  from  an  authoritative  source.  A  transparent  and  traceable  process  is 
one  with  a  clear  explanation  for  how  it  works — in  this  case,  for  how  the  rankings  are 
determined — ^and  one  that  allows  reviewers  to  query  the  process  and  trace  a  decision 
throughout  it.  A  logical  process  is  one  that  agrees  with  the  notions,  experiences,  and 
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understanding  of  the  reviewers.  Such  a  process  can  be  accomplished  by  involving  a 
comprehensive  set  of  subject  matter  experts  (SMEs)  from  the  beginning  and  having  them 
vet  it  throughout.  The  SMEs  should  provide  and  update  the  data  to  ensure  accuracy, 
currency,  and  validity.  Following  these  guidelines,  the  DSE  presented  herein  enables 
stakeholders  to  produce  defensible  decisions  for  meeting  the  capability  gaps  while 
operating  rotorcraft  in  DVE. 

The  method  outlined  in  this  paper  is  based  on  Quality  Function  Deployment  (QFD) 
(Ref  2),  Morphological  Analysis  (Ref.  3),  and  Multi-Attribute  Decision  Making 
(MADM)  (Ref.  4)  techniques.  It  leverages  the  basic  concepts  of  QFD  to  integrate  the 
knowledge  of  SMEs  into  a  quantitative  framework  to  identify  the  most  promising  DVE 
mitigation  technologies.  The  value  of  these  alternatives  is  assessed  using  MADM 
techniques  to  obtain  a  ranked  preference  and  relative  benefit  of  each  alternative.  The 
process  strives  to  be  customizable  by  leveraging  the  most  suitable  methods  and 
techniques  for  the  given  application.  The  methodology  described  in  this  paper  is  an  initial 
iteration  based  on  the  current  understanding  of  the  problem  and  the  stakeholders’  needs; 
it  may  be  modified  at  a  later  stage  based  on  improved  understanding  or  an  evolution  in 
the  requirements. 

Integrating  Expertise  into  the  Decision  Snpport  Environment 

The  process  of  identifying  promising  technology  combinations  cannot  be  conducted 
in  isolation;  it  requires  the  input  of  SMEs  to  (1)  develop  the  structure  of  the  framework 
and  (2)  provide  the  data  necessary  to  perform  the  calculations.  The  success  of  the  process 
is  therefore  predicated  on  the  participation  of  accredited  and  knowledgeable  SMEs.  The 
ultimate  goal  is  to  integrate  the  topic-relevant  information  from  the  group  of  SMEs  into  a 
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morphological  matrix  to  effectively  allow  decision-making  stakeholders  to  produce 
traceable  and  defensible  decisions.  To  accomplish  this  specific  goal,  a  diverse  group  of 
experts  from  government  and  academia  were  invited  to  participate  in  a  series  of 
workshops  to  define  the  problem,  develop  the  framework,  and  provide  their  assessment 
on  how  well  different  technologies  perform  various  functions. 

Overview  of  the  Process 

The  process  is  composed  of  four  high-level  phases.  Defining  the  problem  involves 
understanding  the  underlying  needs,  assumptions,  constraints,  and  viable  sets  of 
alternatives  that  can  be  considered  as  potential  solutions.  Defining  the  DSE  framework 
consists  of  identifying  a  set  of  mappings  linking  the  technology  options  to  the 
overarching  needs.  Collecting  and  integrating  data  involves  not  only  the  process  of 
querying  the  SMEs,  but  also  combining  their  know-how  into  a  mathematical  framework 
that  properly  captures  their  intent.  Leveraging  this  framework  is  the  final  step  by  which 
all  the  work  expended  throughout  the  process  produces  actionable  and  defensible  results. 
The  knowledge  gained  at  this  final  stage  can  inform  updates  to  the  other  steps  and  can 
further  strengthen  and  expand  the  framework. 

Defining  the  Problem 

Oftentimes,  rapidly  conducting  this  first  step  of  the  process  may  seem  desirable  as 
expending  effort  here  may  not  appear  to  be  contributing  to  any  progress  towards  an 
answer.  However,  taking  the  time  to  fully  define  the  problem  is  the  most  important  step 
and  helps  flush  out  the  true  complexity  as  early  as  possible.  This  difficult  step  can  lead  to 
tedious  arguments  about  the  nuances  of  what  the  task  at  hand  involves,  e.g.,  what  is  and 
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is  not  beyond  the  scope,  what  is  the  true  underlying  need.  However,  during  this  step,  tacit 
assumptions  about  the  problem  are  made  explicit  and  an  integrated  understanding  of  the 
problem  can  be  formalized.  The  result  of  the  problem  definition  step  may  appear  obvious 
and  even  trivial  in  hindsight,  yet  it  is  important  to  remember  that  the  new  definition  of  the 
problem  most  likely  differs  from  the  original  form.  Additionally,  the  problem  definition 
may  evolve  as  the  problem  itself  is  further  refined  through  the  process  execution,  but 
ideally,  these  changes  will  not  considerably  impact  prior  findings. 

In  this  particular  application,  the  TRADOC  capability  gaps  for  helicopters  operating 
in  DVE  were  recognized  as  a  defensible  set  of  goals.  Nonetheless,  the  problem’s 
definition  had  to  be  further  refined  in  order  to  clarify  the  true  need  communicated  by 
each  of  the  capability  gaps.  Fig.  1 1  depicts  and  describes  the  eight  gaps  selected  as  the 
highest  level  objectives  along  with  the  inherent  risk  in  not  bridging  that  gap  and  the 
current  capability  to  do  so. 

The  next  step  is  to  bound  the  set  of  alternatives  that  can  be  used  to  mitigate  these 
gaps.  In  this  case,  the  scope  of  alternatives  was  not  limited  by  aircraft  type  (i.e.  the  intent 
was  to  remain  aircraft  agnostic).  The  types  of  feasible  individual  technologies  were  then 
categorized  into  three  main  groups:  Flight  Controls,  Real-time  Sensors,  and  Cueing 
Systems.  Flight  Controls  contains  all  the  technologies  that  can  be  incorporated  into  a 
rotorcraft  to  facilitate  its  operation  and  impact  the  stability  and/or  control  of  the 
rotorcraft.  Real-time  Sensors  includes  all  potential  sensors  that  can  organically  gather 
data  about  the  rotorcraft  and  its  surroundings.  The  term  ‘organic’  implies  that  the  sensors 
cannot  be  dependent  on  transmitted  data  from  other  sensors  or  databases,  except  when 
using  shared  location  information,  e.g..  Blue  Force  Tracker.  Cueing  contains  all  the 
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technologies  for  displaying  and  conveying  information  to  the  pilot,  including  the  various 
modes  for  transmitting  information  (e.g.,  visual,  aural,  tactile)  and  the  symbology  used 
for  representing  this  information.  In  each  group,  all  the  technologies  are  defined  in  a 
system  and  vendor  agnostic  fashion. 


Low  Speed/Hover  Operations 


Low  Lev^l  Flight 


Landing 


Oust  Enviroftmtnt  - 


Mwiln  MiiVv  DumI  Ctipiiwwisrt 


31 


Mooni«ssN^t  HMvy 
Dim  ^ivIronnMKit 


L*ndtngPojnt{LP) 


Extremely  High  Risk 

Extremely  High  Risk 

Extremely  High  Risk 

[  No  Current  Capability 

No  Current  Capability 

Current  Partial  Capability 

RtskttndcutTentcSfMtitMyaHMMfmnWAfebSMdonthcPMA  R.»kandcurr«mcap«MAy  ktMianwnuaretwMAOtiftwpNA  RMkvnd  cunant  oafwtNHtyaaaaMimnnar*  based  on  thsFNA 


GAP  4 

1 

GAPS 

1 1 

GAP  6  1 

Low  Level  Cruise  Flight 

Nap  of  the  Earth  Contour  FHght 

Current  Partial  Capability 


Current  Partial  Capability 


Current  Partial  Capability 


R»h  and  curreni  capaWHy  a»»e#(nntn«»  v*  based  on  ih*  PNA  ftaK  and  current  capaMny  asMssmente  are  based  on  the  PNA  Rak  and  current  cepabHity  aesassmenw  ere  based  on  the  FNA 


Ground  Taxi/Low  Speed  Hover 


GAPS 


Low  Speed/Hover  Operations 


Moonisss  H«svy  Dust 
EntHronment 


High  Risk 

No  Current  Capability 


RtsH  end  current  capobiiity  assessmeme  are  based  on  the  FNA 


Current  Partial  Capability 

Risk  and  cuttent  capabiKry  assessments  are  based  on  dts  FNA 


Fig.  11.  Summary  of  the  TRADOC  capability  gaps  for  rotorcraft  operations  in  DVE 
(distances  &  times  removed  for  OPSEC  reasons). 
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Defining  the  Framework 

The  framework  provides  the  basis  for  the  organization  of  the  data  as  it  is  this  structure 
that  defines  how  the  data  will  be  integrated  and  manipulated.  The  concepts  presented  here 
were  inspired  by  QFD,  a  method  for  translating  qualitative  and  subjective  needs  into 
quantifiable  and  measureable  concepts. 

The  prior  phase  identified  the  capability  gaps  as  the  highest  level  goals  and  a  series  of 
DVE  mitigation  technologies  as  the  set  of  viable  alternatives  that  can  be  integrated  to 
meet  these  goals.  The  next  task  consists  of  identifying  a  set  of  mappings  that  allow  the 
stakeholders  to  trace  the  contribution  of  each  alternative  to  the  mitigation  of  the 
TRADOC  DVE  gaps.  Since  the  technologies  interact  with  one  another,  their 
contributions  cannot  be  assessed  independently  and  none  provides  a  complete,  feasible 
solution. 

To  provide  a  better  mapping  between  broad  gaps  and  specific  technologies, 
intermediate  steps  are  required.  The  first  consists  of  decomposing  the  gaps  down  into  the 
independent  functions  that  must  be  achieved  to  bridge  all  gaps.  The  next  consists  of 
comparing  the  technologies  and  capturing  all  pertinent  differential  functionalities  among 
them.  Finally,  the  two  sets  of  functions  identified  in  these  steps  are  united  and  refined 
over  multiple  iterations.  Similar  functions  should  be  identified  and  specific  differences 
reconciled. 

The  association  between  gaps  and  functions  is  modeled  as  how  critical  achieving  a 
given  function  is  in  fulfilling  a  particular  capability  gap.  The  intent  here  is  to  capture  how 
much  of  a  dissatisfaction  of  the  gap  exists  if  a  singular  function  is  not  achieved,  e.g.,  even 
if  all  but  one  of  the  functions  that  are  required  to  satisfy  a  gap  are  fully  achieved,  the  gap 
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still  may  not  be  achieved.  This  process  is  risk  averse,  as  it  avoids  any  compensation  from 
achieving  most,  but  not  all,  of  the  pertinent  functions.  The  mathematical  form  of  the 
satisfaction  of  a  capability  gap  is  then  formulated  as  the  complement  of  the  maximum 
weighted  dissatisfaction  of  the  functions  as  illustrated  by  Equation  1  below: 

Satisfaction  of  Capability  Gapj  =  1  —  maXj  ^Cij  x  (1  —  (1) 

where  Cij  is  the  criticality  of  function  i  to  capability  gap  j  and  s/  is  the  level  of  satisfaction 
of  said  function  as  mathematically  defined  below. 

Similarly,  the  relationship  between  technology  and  function  must  capture  how  well 
different  technological  options  achieve  the  functions  in  a  specified  environment.  For  this 
application,  a  specific,  commonly  encountered  environment  was  agreed  upon  so  that  the 
elicited  SME  responses  were  formulated  around  a  common  baseline.  The  question  posed 
to  each  SME  is  then  how  well  each  technology  achieves  each  function  in  a  heavy 
sand/dust  environment  at  night.  The  satisfaction  of  a  function  i  (s,)  can  be  calculated  by 
identifying  the  enabled  technology  that  satisfies  the  function  to  the  highest  degree,  as 
defined  below: 

Si  =  maxfe(5i,fe  X  tfc)  (2) 

where  s^k  is  the  satisfaction  of  function  i  by  technology  k  and  4  is  a  Boolean  variable  that 
indicates  whether  technology  k  has  been  selected,  i.e.,  its  value  is  1  when  technology  k  is 
selected,  and  0  when  it  is  not. 

These  functional  satisfactions  can  be  corrected  for  other  environments  through  the 
use  of  a  sensor-to-environment  mapping  table  as  the  environment  primarily  affects  the 
sensors.  The  relative  performance  of  each  sensor  class  is  presented  in  Table  1.  Using  this 
mapping,  the  performance  of  the  sensors  can  be  corrected  for  a  given  environment  of 
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interest  by  multiplying  the  satisfaction  of  a  sensor  related  function  by  their  performance 
multiplier.  For  example,  when  looking  at  snow,  the  performance  of  EO/IR  sensors  will 
remain  unchanged,  but  the  performance  of  Radar  and  LIDAR/LADAR  will  be 
diminished  slightly. 


Table  1.  Sensor  performance  in  each  DVE  of  interest. 


Environment 

EO/IR 

LIDAR 

MMWR 

Smoke 

M+ 

M+ 

N 

Sand/Dust 

N 

N 

N 

Fog 

M+ 

M+ 

N 

Rain 

N 

M- 

M- 

Clouds 

L+ 

M+ 

N 

Snow 

N 

L- 

L- 

Smog 

M+ 

M+ 

N 

Legend:  H+:  Significant  Improvement,  M+:  Moderate  Improvement,  L+:  Minor  Improvement, 

N:  No  Improvement,  L-:  Minor  Detriment,  M-:  Moderate  Detriment,  H-:  Significant  Detriment 

The  overall  structure  of  the  process,  i.e.,  the  framework,  is  depicted  in  Fig.  12.  It 
describes  how  six  different  pieces  of  information  are  integrated  to  generate  viable 
combinations  of  technologies  and  trace  their  contribution  to  bridging  the  capability  gaps. 

1 .  A  prioritization  of  the  gaps  using  pairwise  comparisons. 

2.  The  mapping  between  functions  and  capability  gaps. 

3.  A  prioritization  for  functions  based  on  the  first  two. 

4.  A  mapping  between  technologies  and  functions. 

5.  A  compatibility  matrix  for  each  technology. 

6.  A  set  of  parameters  that  characterize  the  different  technology  alternatives. 
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The  final  product  is  a  prioritized  set  of  technology  combinations.  With  the  exception 
of  products  2  and  4  which  were  defined  previously,  each  one  of  these  is  described  in 
detail  below. 


/  Pairwis«  \ 
C^mp^risDH  \ 

^Prioritization  of  Gaps 

Capability  Gaps 

i 

How  critical  is  it 

s 

d 

o 

tg 

ifi 

fl 

.2 

to  achieve  a  given 

d 

d 

bu 

Q 

c 

function  to  the 

T3 

s 

fulfillment  of  the 

gap?  ^ 

o 

X 

Pu 

RESULT 

A  Prioritized  List  of 
Compatible  DVE 
Mitigation 
Technology  Sets 


PROCESS  RATIONALE 

By  creating  a  traceable  set  of  mappings  between 
Technologies  and  a  set  of  Prioritized  Capability 
Gaps,  we  can  postulate  an  efficient  and 
defensible  experimental  design  for  potential 
DVE  Mitigation  Technologies 


Prioritized  Functions 

Technologies 

o 

How  well  can 
each  technology 
alternative 
perform  each 
function? 

Fig.  12.  Overview  of  the  technology  prioritization  process. 

It  is  clear  that  not  all  the  gaps  are  equally  important,  as  they  impact  different  MTEs 
and  involve  different  levels  of  risk.  The  problem  of  quantifying  how  important  each  one 
is  relative  to  the  others  is  not  trivial,  e.g.,  how  can  a  SME  ascertain  that  Gap  1  is  15% 
important  and  Gap  2  is  12%  important.  Nonetheless,  pairwise  comparison  can  be  used  to 
compare  each  one  on  a  relative  scale  and  obtain  a  normalized  importance  for  each  by 
synthesizing  these  relative  relations.  The  process  is  similar  to  the  one  used  in  the  Analytic 
Hierarchy  Process  (AHP)  and  relies  on  computing  the  principal  eigenvector  of  the 
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weighted  adjacency  matrix.  The  process  and  the  importance  of  this  vector  are  explained 
by  T.  L.  Saaty  (Ref.  5)  in  his  seminal  paper.  Furthermore,  the  consistency  of  the  votes 
can  be  mathematically  computed  to  ensure  that  the  pairwise  comparisons  are  sensible, 
e.g.,  if  the  SME  specifies  that  A  is  more  important  than  B,  and  that  B  is  more  important 
than  C,  then  A  must  be  more  important  than  C,  otherwise,  the  vote  would  be  inconsistent. 
The  consistency  metric  can  capture  more  refined  levels  of  consistency  across  all  pairwise 
comparisons,  e.g.,  if  A  is  twice  as  important  as  B,  and  B  is  three  times  more  important 
than  C,  then  A  should  be  six  times  more  important  than  C.  The  importance  of  the  gaps  is 
normalized  using  the  1-norm,  meaning  that  their  added  importance  equals  1. 

Having  prioritized  capability  gaps  and  a  mapping  relating  gaps  to  functions  allows  for 
the  prioritization  of  functions.  This  prioritization  can  be  computed  as  the  weighted  sum  of 
the  criticality  of  each  function  using  the  capability  importance  as  the  weights,  as  defined 
below: 

Importance  of  Functioui  =  x  Importance  of  Gap!)  (3) 

where  Cij  is  the  criticality  of  function  i  to  capability  gap  j. 

The  last  two  products  determine  the  technology  specific  parameters  that  limit  the 
ability  to  select  specific  combinations.  The  compatibility  matrix  captures  which 
technologies  can  be  integrated,  limiting  the  number  of  combinations  the  stakeholder  can 
select.  For  example,  if  a  particular  display  cannot  represent  a  certain  symbology,  that 
information  can  be  captured  in  the  compatibility  matrix  and  the  combination  of  these  two 
technologies  is  filtered  out  of  the  pool  of  viable  combinations.  The  sixth  and  final  product 
characterizes  the  cost  of  the  technology,  whether  it  may  be  monetary,  based  on  the  notion 
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of  risk,  or  based  on  the  time  it  would  take  for  that  technology  to  become  available.  These 
characteristics  serve  as  filters  for  the  stakeholder. 

Collecting  and  Integrating  Data  into  the  Framework 

The  data  powering  the  framework  was  acquired  during  workshops  and  using  surveys. 
Two  workshops  were  facilitated  at  GTRI  with  the  participation  of  SMEs  from 
government  and  academia.  The  workshops  served  as  an  opportunity  to  discuss  as  a  group 
different  ideas  and  concepts  and  define  the  questions  and  the  set  of  votes  that  would  be 
elicited.  The  actual  voting  was  done  offsite  through  the  use  of  surveys  to  maximize  the 
efficiency  of  the  group.  The  results  were  statistically  studied  to  identify  areas  of  scarcity 
and  contradiction  of  votes.  Due  to  the  wide  variety  of  disciplines  involved,  the  SMEs 
were  asked  to  vote  on  a  mapping  only  if  they  were  familiar  and  comfortable  with  the 
subject  matter.  Therefore,  not  all  SMEs  addressed  all  the  technologies  which  led  to 
certain  mappings  with  a  scarce  number  of  votes.  To  remedy  this  situation,  additional 
SMEs  with  expertise  in  these  areas  were  invited  to  share  their  knowledge.  In  a  few  cases, 
there  were  statistical  disagreements  between  the  SMEs,  but  each  one  was  reviewed  and 
resolved  to  ensure  the  data  in  the  framework  is  accurate  and  pertinent.  Disagreements 
such  as  these  often  lead  to  a  better  understanding  of  the  problem,  as  they  highlight 
different  points  of  view.  Additionally,  disparity  could  be  an  indication  that  the  mapping  is 
not  properly  defined  or  that  it  confounds  various  effects  that  should  be  separated  and 
addressed  independently. 


Leveraging  the  Framework 
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The  framework  defined  in  the  previous  section  can  be  leveraged  to  determine  the  best 
combinations  of  flight  control,  cueing,  and  sensor  technologies  to  best  address  the  DVE 
problem.  The  best  technology  combinations  can  be  determined  by  using  a  cascading  QFD 
approach  to  link  the  capability  gaps  to  the  individual  technologies  and  then  by  using  the 
morphological  matrix  to  define  all  possible  feasible  technology  combinations.  The  math 
for  the  cascading  QFD  is  basic  linear  algebra  that  multiplies  the  capability  gap  to  function 
score  matrix,  i.e.,  mapping,  by  the  function  to  technology  score  matrix.  The  following 
equation  summarizes  the  cascading  QFD  approach; 

f  =  TF  x{FC  X  CW)  (4) 

where  T  is  the  vector  of  individual  technology  scores,  TF  is  the  technology  to  function 

scoring  matrix,  FC  is  the  functions  to  capability  gaps  mapping  (Table  2),  and  CW  is  the 
vector  of  capability  gap  weights. 
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Table  2.  Functions  to  capability  gaps  mapping  (numbers  removed  for  OPSEC  reasons). 


Function  Name  \  Capability  Gap  Number 

I 

II 

in 

IV 

V 

VI 

vn 

vm 

Determine  Location  of  Hazards  at  ?  m 

H 

M 

M 

X 

M 

X 

H 

X 

Detennine  Location  of  Hazards  witbin  ?  m  of  landing  point 

N 

N 

M 

N 

N 

N 

N 

N 

Determine  Location  of  Hazards  witbin  flight  path 

L 

H 

H 

X 

M 

X 

H 

X 

Detennine  Location  of  Forward  Flight  Hazards  at  ?  m 

N 

N 

N 

M 

H 

M 

N 

N 

Determine  Location  of  Forward  Flight  Hazards  at  ?  m 

N 

X 

X 

M 

X 

M 

X 

X 

Detennine  Location  of  Forward  Flight  Hazards  at  ?  m 

N 

H 

N 

M 

N 

N 

N 

N 

Determine  location  of  terrain  and  airborne  hazards  within  ?®  of  the 
aircraft  up  to  ?  X*m 

N 

X 

X 

M 

X 

H 

X 

X 

Detennine  location  ci  terrain  and  airborne  hazards  within  ?**  of  the 

N 

N 

N 

H 

N 

N 

N 

N 

X 

aircraft  up  to  ?  Nm 

Display  Hazards 

H 

L 

H 

H 

H 

H 

H 

Provide  Cueing  of  Aircraft  State 

H 

N 

M 

N 

N 

N 

M 

H 

Provide  Precision  Guidance 

N 

X 

H 

X 

M 

X 

L 

M 

Convey  Collision  Warning 

N 

N 

L 

L 

N 

L 

N 

N 

Com*e\'  Graduated  Collision  Warning 

N 

H 

X 

L 

X 

L 

X 

X 

Detennine  LZ  Slope 

N 

N 

M 

N 

N 

N 

N 

N 

Display  LZ  Slope  (Greater  than  i  ?'") 

N 

X 

M 

X 

X 

X 

X 

X 

Provide  Pitch^U  Stability  in  Hewer  and  Low>Speed  Flight 

H 

N 

H 

N 

N 

N 

H 

H 

Pro^fide  Pitch  Roll  \laneuverability  in  Hover  and  Low-Speed  Flight 

H 

X 

M 

X 

X 

X 

H 

H 

Provide  Y  aw  St^ility  in  H(wer  and  Low -Speed  Fli^t 

M 

N 

M 

N 

N 

N 

M 

N 

Pro\'ide  Y aw  Maneuverability'  in  Ho\*er  and  Low-Speed  Flight 

L 

X 

L 

X 

X 

X 

L 

X 

Provide  V ertical  Stability  h  Hover  and  Low^ -Speed  Flight 

M 

N 

L 

N 

N 

N 

M 

N 

Pren'ide  Vertical  Maneuverability  in  Ho\’er  and  Low -Speed  Flight 

M 

X 

M 

X 

X 

X 

M 

X 

Provide  Position  Control  in  Hover  and  Low -Speed  Flight 

M 

N 

M 

N 

N 

N 

H 

H 

Pro\*ide  Maneu\*eiability  in  Hosier  and  Low-Speed  Flight 

L 

X 

M 

X 

X 

X 

H 

H 

Provide  Pitch/Roll  Stability  in  Forward  Flight 

N 

N 

N 

N 

M 

H 

N 

N 

Pro\’ide  Pitch  Roll  Maneuverability'  in  Forward  Flight 

N 

X 

X 

X 

H 

M 

X 

X 

Provide  Y aw  Stability  in  Forward  Fli^t 

N 

N 

N 

N 

M 

H 

N 

N 

Provide  Yaw  Maneuverability  in  Foiw'ard  Flight 

N 

X 

X 

X 

H 

M 

X 

X 

Provide  V ertical  Stability  in  Forw^ard  Flight 

N 

N 

N 

N 

M 

H 

N 

N 

Pro\'ide  V  ertical  Maneuverability'  in  Foirx'ard  Flight 

N 

X 

X 

X 

H 

M 

X 

X 

Augment  Stability  in  Forward  FHght 

N 

N 

N 

N 

M 

H 

N 

N 

Augment  Maneuverability'  in  Fon\  ard  Flight 

X 

X 

X 

X 

H 

M 

X 

X 

Determine  Absolute  Positioning  of  the  Aircrait 

M 

N 

M 

M 

M 

M 

M 

M 

Legend:  H;  Significant  Impact  M:  Moderate  Impact  L:  Minor  Impact  TC:  Xo  Impact 


Given  the  vector  of  individual  technology  scores  from  Equation  4,  the  morphological 
matrix  can  then  be  used  to  determine  the  best  combination  of  flight  control,  cueing,  and 
sensor  technologies.  A  feasible  technology  combination  is  generated  by  picking  one 
technology  from  each  of  the  subgroups  and  is  scored  by  summing  eaeh  of  the  scores  from 
the  chosen  technologies.  Using  a  morphological  matrix  to  explore  all  possible,  feasible 
technology  combinations  generally  results  in  an  intractable  number  of  alternatives. 
Consequently,  it  is  preferable  to  use  an  algorithm  to  find  the  “best”  combination  of 
alternatives.  Oftentimes,  evolutionary  optimization  algorithms  are  used  for  this  purpose. 


24  DELLERT,  BIVENS,  BALESTRINI-ROBINSON,  ZENTNER,  HOPKINS 


but  in  this  case,  the  technology  combination  scoring  function  is  relatively  simple  and  the 
alternative  space  is  relatively  small  (approximately  56  million  combinations),  so  it  is 
possible  to  fully  search  the  space  to  determine  the  top  10  alternatives.  Fig.  13  presents  the 
top  10  technology  combinations  identified  by  this  search  and  provides  insight  into  which 
technologies  are  categorically  preferred  by  the  algorithm  and  which  are  interchangeable. 


e.g..  Position  Hold  in  Hover/Low  Speed  Flight  and  LIDAR  imaging  modes,  respectively. 


Category 

Group 

Subgroup 

Technology  Option 

Top  10  Technology  Portftjlios 

P  1|P2|P  3lP4|P5|P6lP  7|P  8|P9|P  10 

lA 

Ho\er  Lew-Speed  ' 

Position 

Hoter  Hold  (HvrHId) 

Position  Hold  (PH) 

0000000000 

1111111111 

Yaw 

Yaw  Rate  Command  (RC) 

Direction  Hold  (DH) 

0000000000 

1111111111 

Vertical 

Vertical  Rate  Command  (RC) 

Height  Hold  (HH) 

1111111111 

0000000000 

c 

o 

o 

JC 

Pitch/Roll 

Rate  Cemnmand  /  Attitude  Hdd  (RCAH) 

Rate  Command  (RC) 

Translational  Rate  Command  (TRC) 

Attitude  Command  /  Attitude  Hold  (ACAH) 

0000000000 

0000000000 

0000000000 

1111111111 

Foiward  Flight 

Yaw 

Yaw  Rate  Command  (RC) 

Direction  Hold  (DH) 

1111111111 

0000000000 

Cueing 

Vertical 

Altitude  Hold  (HH) 

Vertical  Rate  Command  (RC) 

0  0  0  0  0  0  1  1  1  1 

1  1  1  1  1  1  0  0  0  0 

Pitch/Roll 

Rate  Command/Attitude  Hold  (RCAH) 

Rate  (Command  (RC) 

Attitude  Command/Attitude  Hold  (ACAH) 

0000000000 

0000000000 

1111111111 

Pitot  Vehicle  interface 

Visual 

Panel  Mounted  Display 

Narrow  FOV  with  Head  Tracking  Kspiay 
Narrow  FOV  VMthout  Head  Tracking  Display 
Wide  FOV  with  Head  Tracking  Display 

VWde  FOV  without  Head  Tracking  Dsplay 

1111111111 

0000000000 

0000000000 

0000000000 

0000000000 

Audio  Projection  Mode 

Mono  Audio 

Stereo  Audio 

3-D  Audio 

0000000000 

0000000000 

1111111111 

me  Sensors 

Terrain/  Obstacle  tmageiy 

Scene  Imagery 

Fused  Imagery 

Enhanced  (Sensor  Only) 

Synthetic  (Database  Only) 

1111111111 

0000000000 

0000000000 

Symbology 

3D  Terrain  Conformal  Symbology 

Conventional  2D  Aircraft  PFR  Symbology 

1111111111 

0000000000 

UDAR/LADAR 

Detection  Mode 

Geiger  Mode  Detection 

Heterodyne  Detection 

Direct  Detection 

0  0  1  1  0  0  0  1  0  1 

1  0  0  0  1  0  0  0  1  0 

0  1  0  0  0  1  1  0  0  0 

Imaging  Mode 

No  UDAR/LADAR 

Scanning  UDAR/LADAR 

Flash  UDAR/LADAR 

0000000000 

0  0  0  1  1  1  0  0  0  1 

1  1  1  0  0  0  1  1  1  0 

Passive/Acdve  kna^ng 

No  Imager 

Millimeter  Wave  Imager 

Long  Wa\«  Irrirared  Imager 

Medium  Wave  Infrared  imager 

Short  Wave  Infrared  tmager  (Illuminated) 
Vistble/Near  Infrared  Imager 

oooooooooo 

0000000000 

1111111111 

oooooooooo 

oooooooooo 

oooooooooo 

n 

Radar 

Frequency  | 

No  Radar 

Ku  Band  (12  to  18  GHz) 

X  Band  (8  to  12  GHz) 

W  Band  (75-110  GHz) 

Ka  Band  (26.5  to  40  GHz) 

oooooooooo 

oooooooooo 

oooooooooo 

1111111111 

oooooooooo 

o: 

Mode  1 

Real  Beam  Radar 

Synthetic  Aperture  Radar 

1111111111 

oooooooooo 

Precision  Geopositioning 

Inertial  Navigation  System  (INS) 

Global  Positioning  System  (GPS) 

Embedded  GPS/INS  (EGI) 

oooooooooo 

oooooooooo 

1111111111 

Fig.  13.  Top  10  technology  portfolios  identified  by  the  study. 
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The  framework  can  be  queried  in  a  myriad  of  different  ways  to  discover  new 
information  that  was  not  apparent  in  the  original  data.  This  requires  a  certain  degree  of 
creativity  and  a  high  degree  of  mathematical  rigor  to  avoid  drawing  inappropriate 
conclusions.  Below  is  a  description  of  additional  results  that  can  be  gleemed  from  the 
DSE. 

In  addition  to  identifying  the  preferred  individual  technologies,  or  combinations 
thereof,  the  framework  can  give  an  indication  of  how  much  each  category  of  technologies 
contributes  to  the  overall  bridging  of  the  gaps.  This  is  important  because  it  prioritizes  the 
groups  of  technologies  and  informs  the  stakeholders  as  to  which  solution  sets  should  be 
researched  further.  Analyzing  the  maximum  value  of  each  technology,  as  calculated  by 
deploying  the  capability  gap  importance  through  the  two  mapping  matrices,  it  is  possible 
to  estimate  how  important  each  technology  alternative  is.  Using  these  values  for  each 
sub-category,  one  can  estimate  the  average  maximum  contribution  from  each  sub-group 
for  each  category,  i.e.,  an  indication  of  how  much  each  group  of  technologies  can 
contribute  to  bridging  the  eight  capability  gaps.  Fig.  14  depicts  the  results  showing  that 
46%  percent  of  the  contribution  is  provided  by  the  sensors,  while  39%  is  provided  by  the 
cueing  systems,  and  15%  by  the  flight  controls.  This  agrees  with  the  concepts  defined  in 
ADS-33  (Ref.  1),  whereby  the  addition  of  sensors  and  cueing  would  improve  the  Usable 
Cue  Environment  (UCE)  and  improved  flight  controls  would  attain  the  required  level  of 
handling  qualities.  In  addition  to  this  conclusion,  this  figure  shows  that  no  single  group  of 
technologies  can  address  all  the  gaps.  The  DVE  solution  must  leverage  technologies  from 
each  one  of  these  groups. 
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It  must  be  pointed  out  that  many  assumptions  and  assertions  were  identified  by  the 
subject  matter  experts  on  the  team  prior  to  their  “scoring”  iteration.  These  were  necessary 
in  order  to  present  coherent  scenarios  and  metrics  that  could  be  scored  without  bias. 
Additionally,  the  TRADOC  capability  gaps  utilized  were  the  basis  that  the  experiment 
was  formed  around;  the  gaps  are  somewhat  “imaging  centric”  and  a  revised/expanded  set 
of  capability  gaps  would  most  assuredly  result  in  different  outcomes/percentages.  Finally, 
it  must  be  noted  that  all  elements  of  the  DVE  system  material  solution  (the  3  legs  of  the 
stool)  are  essential  as  no  one  or  two  leg(s)  can  provide  the  objective  endstate  of  an 
aircraft  being  able  to  expand  its  operational  maneuver  space  by  executing  combat 
operations  in  degraded  visual  environments  with  Level  1  handling  qualities  throughout 
the  flight  envelope. 

Flight 

Controls 

Sensors 
46% 


Fig.  14.  Average  maximum  contribution  of  each  technology  sub-category. 

Conclusions 

The  paper  outlines  the  development  of  a  framework  to  support  decision-makers 
tasked  with  developing  an  experimental  plan  for  DVE  mitigation  technologies.  The 
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process  traces  the  alternatives  to  the  highest  level  goals,  and  allows  decision  makers  to 
not  only  obtain  a  ranked  set  of  combinations  to  test  based  on  how  well  they  address  these 
high  level  needs,  but  it  allows  them  the  transparency  and  traceability  of  querying  the 
framework  and  tracking  the  operations  performed  by  the  DSE.  The  data,  and  the  structure 
of  the  process,  was  not  developed  in  isolation,  but  with  the  assistance  and  guidance  of  a 
broad  group  of  SMEs  from  government  and  academia.  This  ensured  that  their 
experiences  and  opinions  were  reflected  not  only  in  the  data  they  provided  to  populate  the 
DSE,  but  also  in  its  architecture  and  logic  and  ultimately,  the  results  generated.  The  final 
product  is  a  logical  series  of  calculations  that  mathematically  capture  the  SMEs’  rationale 
and  allows  decision  makers  to  query  the  DSE  without  having  to  independently  query  the 
SMEs  on  every  combination  of  technologies.  In  essence,  the  environment  serves  as  a 
surrogate  of  the  SMEs’  combined  experience  and  know-how  on  the  subject.  This  makes 
the  process  rational,  and,  combined  with  the  ability  of  the  framework  to  incorporate 
relevant  information,  makes  the  overall  decision  defensible. 
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